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Image  processing  often  requires  die  application  of  a  two  dimensional  operator  out  the  image.  Imaging 
devices  usually  scan  the  image  along  a  one-dimensional  path,  and  produce  a  stream  of  data  points.  In  such 
cases,  an  intermediate  storage  is  required,  to  interface  between  the  one-dimensional  scanning  and  the  applica¬ 
tion  of  the  two-dimensional  operator. 


In  this  paper,  the  interface  problem  will  be  examined.  In  particular,  we  shall  determine  the  minimum 
si/e  required  of  the  intermediate  storage.  l;or  a  square  image  of  si/e  rr ,  and  a  square  opet  atot  of  si/e  m'a  the 
minimum  intermediate  storage  is  shown  to  be  n  •  (m  -  -  I).  l  or  small  si/e  operators  the  intermediate  storage 
mat  not  add  substantial!)  to  the  overall  size  of  the  system,  l  or  example,  a  teceni  two-dimensional  CVI) 
scanner  developed  by  Texas  Instruments  (Hall  &  Aw  trey.  l‘)71))  incorporates  the  intermediate  storage  required 
for  a  3  *  3  operator  on  the  scanner  chip  itself.  I -'or  larger  operators,  however,  the  si/e  of  the  intermediate 
storage  becomes  significant.  Tor  example,  a  recent  CC1 )  convolution  chip  constructed  by  I  lughes  is  designed 
to  support  .i  20  *  20  operator  (Nudd  <7  <//..  f)7‘)),  It  is  intended  to  be  applied  to  images  of  KKKI  points  on  a 
side,  in  which  case  an  intermediate  storage  of  2500(1  image  points  would  be  required.  In  such  cases  the  si/e  of 
the  system  is  determined  almost  entirely  by  the  intermediate  storage  requirement.  ( liven  the  theoretical  lower 
bound  on  the  si/e  of  this  storage,  it  is  suggested  that  new  kinds  of  imagers  would  be  required  for  systems  for 
w  Inch  si/e  is  a  dominant  consideration. 


2.  I  'sing  a  serpentine  delay  line  of  si/e  n  (m  I ) 

A  simple  method  for  interfacing  a  one-dimensional  scanner  to  a  two  dimensional  operator  is  shown  in 
f  igure  I.  The  camera  c  scans  the  image  (assumed  to  be  a  square  of  n/  points)  row  by  row.  Hie  intensity  values 
measured  by  the  camera  are  fed  into  a  soipenlinc  delay  line  (S  in  figure  I),  consisting  of  m  I  rows,  each 
of  length  n.  I  lie  serpentine  delay  line  is  tapped  in  m  I  positions  that  are  connected  to  the  two  dimensional 
operator  M  (of  si/e  m’).  In  addition,  there  is  a  direct  line  from  the  camera  to  l\|.  so  that  there  arc  a  total  of  m 


.1 


input  lines  to  M 

After  the  first  m  —  I  rows  have  been  scanned,  they  till  the  intermediate  storage  S,  and  the  first  column 
of  these  rows  resides  in  the  rightmost  column  of  S.  ready  to  he  delivered  to  INI.  The  scanner  then  reads  point 
(m.l)  in  the  image,  and  the  first  column  of  m  points  (point  (1.1)  to  (m.l))  is  transferred  to  M.  I’oint  (m.l) 
also  enters  the  delay  line,  shifting  each  point  in  S  by  one  position.  The  point  in  the  lower-right  corner  of  S 
is  dropped  from  the  memory  and  lost.  Whenever  a  new  point  is  now  read  In  the  scanner,  a  column  of  m 
points  is  transferred  to  M.  After  hav  ing  scanned  m  points  from  the  m"1  row.  the  entire  upper-left  septate  of  m  ’ 
points  resides  in  M.  and  the  operator  is  applied  to  it.  When  (lie  scanning  of  the  entire  image  is  completed,  the 
operator  has  been  applied  to  all  the  squares  of  si/e  m2  in  the  image. 

The  intermediate  storage  S  in  this  scheme  is  required  to  hold  n  (m  -  I)  image  points.  The  next  section 
will  establish  dial  this  requirement  cannot  be  reduced.  Tven  if  one  uses  arbitrary  scanning  pattern  (e.g..  the 
image  may  be  divided  into  sub-blocks,  and  each  block  scanned  and  operated  upon  separately),  and  arbitrary 
interconnections  between  S  and  M  (i.c..  M  may  have  access  to  all  the  points  in  S  ).  the  minimal  intermediate 
storage  would  still  be  n  (m  —  l). 
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V  The  minimal  intermediate  storage  is  n  (m  -  1) 

I  ct  the  si/c  of  the  intermediate  storage  he  S.  in  this  part  we  wish  to  show  (hat  S  >  n  (nt  1). 

Noldiion:  We  shall  think  of  the  image  as  a  grid  of  n  *  it  squares,  ri  >  2.  Hath  square  in  the  grid  is  also 
tailed  a  /vw  in  the  image.  A  two  dimension.il  operator  of  si/e  in  ♦  in  is  a  function  of  nr  aigumeiits.  I  he 
funelion  is  to  he  applied  to  every  square  of  si/e  in1  points  iit  the  image.  We  shall  assume  that  the  si/e  of  the 
opcralor(m)  is  at  most  !j.  filially.  for  a  set  I’ of  points.  ||/’jj  will  denote  the  number  of  points  in  I’. 

(hfiiutions:  Two  distinct  points  in  the  image  are  called  pm  liters  if  they  tan  lie  entered  simultaneously  In 
a  square  of  si/e  m2.  Chat  is.  their  distance  in  the  image  in  either  the  horizontal  or  the  terocal  dilation  is 
at  most  m  points.  Similarly,  the  partners  of  a  set  I’  of  points  are  the  partners  of  all  the  points  in  I',  hut  not 
including  the  points  in  tile  set  I*  itself.  That  is. 


partncrs(l')  U  partners(i)  -  /A  (I) 

»  r 

When  the  scanner  has  read  k  points,  k  >  S.  some  of  the  k  points  are  no  longei  in  the  Mmugc.  and  e.m  no 
longer  he  retrieved.  I  el  I’  he  the  set  of  points  that  have  been  scanned,  hut  are  no  longei  in  die  stoiage.  It  is  not 
difficult  to  see  that  at  this  stage  the  partners  of  the  set  I’  must  reside  in  the  inieimediate  stoiag.e  I  he  leason  is 
that  if  points  p  and  q  are  partners,  then  there  must  exist  a  stage  at  which  both  p  and  q  aie  in  the  mtei  mediate 
storage.  Wo  can  now  obtain  a  lower  hound  on  tile  si/c  of  the  intermediate  stoiage  In  estimating  the  number 
of  partners  of  a  set  of  k  points.  We  shall  establish  the  low  er  hound  on  the  si/e  of  the  mtei  mediate  stoiage  by 
showing  that  for  a  certain  choice  of  k,  the  number  ofpaitiicis  is  at  least  n  (m  I ). 

Proposition  I:  I  et  k  he  the  smallest  integer  such  that  k  '  ■  ",  .  l-or  this  choice  ot  k.  any  set  of  A  points  has 
at  least  n(m  -  I )  partners. 

Proof:  lb  induction  on  m.  the  si/e  of  the  two  dimensional  operator.  Hoi  in  1  the  claim  holds,  since 
the  number  of  partners  is  /ero.  To  prove  tile  induction  step  we  shall  show  that,  in  an  image  of  si/c  n*.  when 
the  si/c  of  the  operator  is  increased  from  m  -  M  «>  m  -  M  ^  I.  (assuming  A/  |  1  •  y ).  the  set  of 'p.utners 


is  increased  hy  at  least  n  new  points.  To  prove  the  induction  step,  we  shall  use  the  following  tw.i  definitions 


Ihyimimn  I:  I  cl  I  he  the  image,  and  P  a  collection  of  point*  in  the  image.  A  neighbor  of  the  set  P  is  a 
point  in  I  •  1’  that  shares  an  edge  or  a  vertex  on  the  grid  with  a  point  in  P. 

Ihjinmon  2:  In  a  grid  of  si/e  n2.  a  set  of  k  points  is  called  tiJintssiMclnmi  the  grid  itself  is  admissible)  if: 

*  n*  <  k  <  n2  for  n  >  2  (2) 

4  4 

*  n2  <  )c  <  |n2  for  n  —  2 

4  4 

Comment:  I  et  P  be  the  set  of  the  k  original  points,  and  all  their  partners  for  an  operator  of  si/e  M.  When 
the  operator  increases  in  si/e  to  M  f  1.  all  the  neighbors  of  P  become  new  partners  of  the  k  original  points. 
I  o  prove  proposition  I  it  will  he  sntlicicnl.  therefore,  to  show  that  the  set  P  has  at  least  n  neighbors.  I  bis  will 
he  the  goal  of  proposition  2. 

Without  loss  of  generality.  we  can  assume  that  the  set  P  is  admissible.  Since  P  contains  the  k  original 
points,  clearly  fl,;  <  ||/’||.  If  ||/’||  >  |n2.  then  the  k  original  points  have  at  least  J n2  —  k  partners. 
Put  since  k  <  I.  and  'j  '>  M  |  I.  it  would  follow  that  the  k  original  points  have  at  least  n  •  rn 

partners.  Proposition  I  would  therefore  hold  for  the  operator  of  si/e  A I  |  I .  and  the  induction  step  would  he 
unnecessary. 

Proposition  2:  In  a  grid  o fn2  points,  any  admissible  set  of  A  points  has  at  least  n  neighbors. 

Proof:  lly  induction  on  it.  the  si/e  of  the  grid.  I  or  »i  2  the  proposition  holds,  since  K  =  1  or  K  =  2, 
and  in  either  case  there  are  at  least  2  neighbors.  We  next  assume  that  proposition  2  holds  for  n  =  N.  and 
prove  it  for  n  ~  N 

I'he  general  structure  of  the  proof  will  be  as  follows,  f  or  convenience  of  reference,  the  A  points  will  be 
called  the  black  points,  and  the  remaining  points  white.  In  these  terms,  we  wish  to  show  that  in  a  grid  of  si/c 
(N  f  I)7  am  set  of  A  points.  \(N  -f  I)2  <  K  <  \(N  f  I  )2.  has  at  least  A'  )  1  white  neighbors.  To  use 
the  induction  hypothesis  we  shall  prove  the  following  claim: 
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Proposition  Il  is  possible  to  remove  a  row  and  a  column  from  die  grid  in  such  a  was,  that  the  resulting 
grid  of  si/e  N 2  w  ill  be  admissible. 

From  the  induction  hypothesis,  in  die  reduced  grid  the  set  of  black  points  has  at  least  N  neighbors.  Hie 
proof  will  be  concluded  by  showing  that  when  the  missing  row  and  column  are  re  introduced,  the  number  of 
neighbors  of  the  black  set  is  increased  by  at  least  one. 

Proposition  .?.  I:  From  an  admissible  grid  of  si/e  ( N  -f-  1)-  it  is  possible  to  remove  a  row  and  a  column,  so 
that  in  the  remaining  grid  of  si/e  N2  the  number  of  black  points  will  be  less  than  \N2. 

Proof:  l  et  a\4-i  denote  the  fraction  of  black  points  in  the  original  grid.  That  is.  the  number  A'  of 

black  points  is  equal  to  a, \  ,  |  ■  ( N  -F  l)2.  Similarly.  a%  is  the  fraction  of  black  points  in  the  reduced  grid.  In 
these  terms,  we  assume  in  proposition  3.1  that  \  <a%  (  i  <  j.  and  wish  to  establish  that  a  \  <  ;j. 

Let  B,j  denote  the  total  number  of  black  points  in  row  i  and  column  j.  BtJ  is  the  (number  of  black  points 
in  row  i)  +  (the  number  of  black  points  in  row  j).  The  difference  between  B:J  and  BtJ  is  that  if  point  (ij)  is 
black,  it  is  counted  twice  in  BtJ  but  only  once  in  B,j.  I  cl  B  ~  ^  ~  SiV  *i 

Ihcrc  are  aN  +  t(N  l)2  black  points  in  die  grid,  and  each  one  is  counted  2 (/V  j  I)  times  in  B. 
therefore: 

B  —  2(/V  -f  I)  oN  ,  ,(N  T  l)2  (il) 

In  B  each  black  point  is  counted  one  additional  time,  compared  with  B.  therefore: 

B  =  B  -nv  +  ,(N  +  l)2  (4) 

There  are  (N  -f  l)2  terms  BtJ  cmitributing  to  the  sum  B.  therefore  die  average  contribution  is: 

(N  T-  l)2  =  2<Jn  *  +  •)  ~  «v  t  i  (5) 

There  must  exist  a  term  BtJ  that  contributes  at  least  the  average  contribution.  I’roposition  3.1  will  therefore  be 
satisfied  if: 

as  n(/V  I)2  -  (2a %  ,  ,(/V  |  l)-aX|l)<  V2 

4 


(6) 


If  this  inequality  docs  not  hold,  then: 


<>\  t  i (N  I)2  -  (2ax  ,  t(N  |  I)  -  as  ,  ,)  >  V2  (7) 

1 

Hut  ihisimpliesus  (  \  >  •(,  which  is  a  contradiction,  since  <a\  |  i  <  •{.  | 

I'liii’osiiiini  In  the  conditions  of  proposition  .1.1  it  is  possible  to  remove  n  row  .ind  ,i  column  so  lh.it  in 
the  reduced  grid  c»\  >■  \. 

I  he  proof  will  he  omitted,  since  it  is  entirely  analogous  to  the  proof  of  proposition  .1.1. 

Comment:  I  he  row  and  column  chosen  in  propositions  .1.1  and  .1.2  are  not  necessarily  the  same.  If.  in 
proposition  .1.1.  a\  >  or  if.  in  proposition  .1.2.  a \  <  j.  then  proposition  .1  is  established.  Proposition 
3  may  still  fail,  however  to  he  satisfied  if  in  proposition  3.1  o\  <  and  in  proposition  3.2  o\  >  ;-j.  It  is 
required  therefore  to  show  that  in  this  case  vve  can  still  remove  a  row  and  a  column  such  that  in  the  reduced 
grid  the  set  of  black  points  is  admissible. 

I  et  the  row  and  column  chosen  in  proposition  3.1  he  row  /  and  column  j.  I  lie  row  and  column  chosen 
in  proposition  3.2  will  he  denoted  row  A  and  column  /.  I  et  us  now  remove  from  the  grid  row  i  and  column  /. 
Since  the  definition  of  an  admissible  set  is  slightly  different  for  N  -  2,  we  shall  start  with  the  case  N  >  3.  In 
this  case,  proposition  3  would  fail  to  he  satisfied  if  either: 

«m  .(n  f  iy2- -».i<  W  (8) 

4 

or: 

ov  ,,(N  \  \y  -  If,,  >  34N 2  (9) 

I  et  us  examine  the  first  case.  We  may  assume  that 

a\  i  i(/V  f  If  -  Ik,  >  V 

4 


(10) 
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(see  the  comment  above),  from  (8)  and  ( 10)  we  conclude  that: 

H,i-lkt>i2N2  (II) 

H,t  —  l hi  is  the  difference  in  the  number  of  black  points  between  rows  /  and  A.  alter  removing  the  l"'  column, 
hence  B,t  -  Ihi  <  A.  This,  together  with  (II).  implies  A  <  2.  in  contradiction  with  the  assumption  that 
A  >  3.  We  conclude  that  for  A  >  3  (8)  is  impossible,  and  the  inequality  «\  |  i(A'  (  I)'  -  B,i  >  ‘A2 
must  hold.  To  include  die  case  A  —  2.  a  similar  argument  establishes  that  n\  (  \(N  f-  l)2  —  B,i  >  \N2 
must  hold  for  A  >  2. 

The  case  in  (9)  can  be  handled  analogously,  yielding 

-  B,t  >  ‘  N2  (12) 

which  again  cannot  hold  for  N  >  2. 

In  conclusion,  the  fraction  a \  satisfies  |  <  n\  <  j  for  A  >  3,  and  <  i»\  <  \  lor  N  2.  I  he 
set  of  black  points  in  the  reduced  grid  is  therefore  admissible.  | 

Proposition  3  has  established  that  a  row  and  a  column  can  be  removed  from  the  original  grid  of  si/e 
(/V  f  l)2,  resulting  in  an  admissible  grid  of  si/e  N .  Irom  the  induction  hypothesis,  in  the  reduced  giid  the  set 
of  black  points  has  at  least  N  neighbors.  To  show  that  the  set  of  neighbors  in  the  original  grid  is  larger  (by  at 
least  one),  we  shall  use  the  following  claim: 

Proposition  4:  proposition  3  can  he  satisfied  by  remov  ing  a  row  and  a  column  that  are  not  entirely  black. 

Proof:  It  is  suDicicnl  to  show  that  proposition  3.1  can  be  satisfied  w  ith  a  row  and  a  column  that  contain 
at  least  one  white  point.  We  can  assume  without  loss  of  generality  that  the  original  grid  contains  the  maximal 
possible  number  of  black  points,  since  if  proposition  3.1  can  be  satisfied  (with  a  row  and  a  column  that  are 
not  entirely  black)  for  a  maximal  number  of  black  points,  it  can  also  be  satisfied  for  grids  with  fewer  black 
points,  l-o  rN  -f-  I  —  2m  (an  even  number),  the  maxima!  number  of  black  points  is  \(N  (  1 )/  I.  for 


/V  T  1  —  2m  -  (-I,  the  maximal  number  is  \(N  -f-  l)2  —  We  shall  examine  the  killer  ease  (the  first  can  be 
established  analogously). 

I  ei  r  be  the  number  of  completely  black  rows  in  the  original  grid,  c  the  number  of  completely  black 
columns.  We  can  assume  that  r  ~>  1 ,  r  >  1,  since  otherwise  proposition  4  follow es  directly  from  proposition 
.1.  Since  «\  j  i  <  j,  either  c  <  v  -j  -  or  r  <  N  j  1  must  hold.  Without  loss  of  generality  we  shall  assume 
that  c  <  N  j - . 

(  he  r  black  rows  and  c  black  columns  contribute  a  total  of 


(r  +  c)(N  f  l)  -  rc 


(13) 


black  points.  I  be  number  of  black  points  in  the  remaining  rows  and  columns  is  therefore: 

\(N +  \f--4~(r  +  c)(N +  l)  +  rc 


(14) 


This  number  of  black  points  is  divided  among  N  -f  1  —  c  columns,  hence  the  av  erage  per  column  (denoted 
by  A)  is: 

!(«+!)'-  j-(r  +  U(N  +  l)  +  rc 

- N+r-c — . — -  "5> 

There  must  exist  a  column  which  contains  at  least  r  -f-  A  black  points.  By  removing  this  column  and  one  of  the 
black  rows  we  remove  at  least: 

(iV+O  +  r-M-l  (16) 

black  points.  It  thcrefo  e  remains  to  establish  that: 


\(N  +  l)2  -  (N  +  r  +  A)  <^2 


(17) 


Inequality  (17)  reduces  to: 


N2  Nc 

4  2  -C>° 


(18) 


lor  r  <  N  ^  1  inequality  (IS)  holds,  therefore  proposition  4  is  established.  | 


Corollary:  using  the  same  argument,  proposition  3.2  can  he  established  with  a  row  and  a  column  that 
are  not  entirely  white.  Combining  the  two  claims,  it  is  straightforward  to  establish  that  proposition  3  cam  be 
satisfied  by  a  row  and  a  column  Unit  are  neither  entirely  black  nor  entirely  white. 

Proposition  5:  The  set  of  black  points  in  die  original  grid  has  at  least  N  -f-  1  neighbors. 

Proof:  l-'rom  propositions  3  and  4,  we  can  remove  a  row  R  and  a  column  C  (neither  entirely  black  nor 
entirely  white)  from  die  original  grid,  and  obtain  an  admissible  grid,  f  rom  the  induction  hypothesis,  the  set  of 
black  points  in  the  reduced  grid  has  at  least  N  neighbors.  We  have  to  show  that  by  re-introducing  R  and  C 
the  number  of  neighbors  increases  by  at  least  one. 

Since  R  and  C  contain  both  black  and  white  points,  they  contain  at  least  one  "internal  neighbor",  i.e.,  a 
while  point  that  borders  a  black  point,  where  both  points  arc  in  R  (J  C.  We  shall  show  that  w  ithout  the  inter¬ 
nal  neighbor  the  number  of  neighbors  in  the  original  grid  is  at  least  N.  hence  the  total  number  of  neighbors  is 
at  least  N  -J-  1.  as  required. 

If  a  given  point  p  is  a  neighbor  of  the  black  set  in  the  reduced  grid,  it  may  or  may  not  remain  a  neighbor 
when  the  missing  row  and  column  tire  rc-introduccd.  l  et  us  examine  first  the  case  where  p  is  not  a  neighbor 
of  the  black  set  in  the  original  grid.  In  this  case  p  must  be  surrounded  in  the  original  grid  by  w  hite  points  (see 
figure  2).  In  this  configuration,  point  q  in  R.  becomes  a  new  neighbor,  since  at  least  one  of  the  points  a,  b ,  c 
must  be  black.  Note  that  q  cannot  be  an  internal  neighbor.  (Comment:  in  f  igure  2.  point  p  borders  the  re¬ 
introduced  column  R.  The  argument  is  unchanged  when  it  borders  C  instead  of  R.  The  case  where  p  borders 
both  R  and  C  requires  a  slight  extension  of  the  argument,  which  w  ill  not  be  elaborated.) 

We  conclude  that  there  exists  a  one-to-one  mapping  front  the  neighbors  in  the  reduced  grid  into  the 
neighbors  in  the  original  grid,  and  that  this  mapping  does  not  include  the  internal  neighbor.  The  number 
of  neighbors  in  the  grid  of  si/e  N  f-  \  h  therefore  at  least  N  This  establishes  the  induction  step  in 
proposition  2.  and  concludes  the  proof  of  propositions  2  and  1.  | 


4.  Reducing  t  lie  si/o  of  the  intemiediiite  storage 


The  lower  hound  established  in  the  preceding  section  implies  that  the  intermediate  storage  may  play  a 
dominant  role  in  determining  the  si/e  of  the  system.  The  intermediate  storage  can.  however,  he  reduced  by 
scanning  the  image  more  than  once.  Suppose,  for  example,  that  v  rows  are  scanned  in  parallel.  The  first  scan- 
path  scans  rows  1  to  v.  the  second  scans  rows  2  to  .s  |  1.  etc.  Inch  image  point  is  scanned  in  this  scheme  ,v 
times.  Using  the  serpentine  delay  line  of  Section  2.  it  is  possible  now  to  connect  v  lines  directly  to  the  two- 
dimensional  operator  M.  The  intermediate  storage  S  will  then  have  only  (»«  .s)  rows,  l  itis  scheme  gives 

a  simple  (though  not  optimal,  in  terms  of  the  storage  si/e)  method  of  reducing  the  si/e  of  the  intermediate 
storage  to  n  •  (rn  —  s).  where  .v  is  the  number  of  times  each  point  is  scanned.  In  particular,  by  having  s  -  m 
the  intermediate  storage  becomes  unnecessary. 
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hkmmleJyement:  I  thank  M  Hr.idy  and  b.  I  lildrelh  for  their  helpful  comments. 


